Pipelined Neural Tree Learning by Error Forward-Propagation

We propose a new parallel implementation of the neural tree feed-forward network architecture that supports eecient evaluation and learning regardless of the number of layers. The neurons of each layer operate in parallel and the layers are the elements of a pipeline that computes the output evaluation vectors for a sequence of input pattern vectors at a rate of one per time step. During the learning phase the desired outputs are presented as additional inputs and the pipeline computes in feed-forward manner the gradients of the errors with respect to the neuron evaluations. Thus it is possible to run diierent gradient descent learning algorithms on the pipeline with a performance comparable to the evaluation algorithm.